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ABSTRACT 

This paper describes the design of our service-oriented architecture to support mobile multiple object tracking augmented 
reality applications applied to education and learning scenarios. The architecture is composed of a mobile multiple object 
tracking augmented reality client, a web service framework, and dynamic content providers. Tracking of multiple real 
objects and retrieval of associated multiple media contents allows more complex augmented reality learning scenarios to 
be constructed that could improve students’ knowledge and learning strategies on a mobile platfonn. It also allows 
students to create their own augmented reality learning environments and select preferences from acquired digital 
contents based on multiple object real scenes. Mobile users are able to request contextual digital contents from web 
service providers to augment these multiple objects in the real world. The digital contents are generally dynamically 
acquired digital media, e.g. 3D models, images, textual descriptive data, metadata, multimedia or even social media data. 
New digital contents for augmenting the real world are acquired through a service-oriented approach by accessing any 
appropriate web services to deliver that content to the augmented learning environment. 
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1. INTRODUCTION 

Augmented reality is a concept for displaying digital contents overlaid on top of real world scenes that can 
enhance remarkably a user’s learning experiences. Augmented reality (AR) utilizes advanced computer 
vision and tracking techniques to recognize markers, images or 3D objects in the real environment and uses 
this information to augment the physical space with computer generated media contents, such as 3D models, 
sound, images, videos, texts, etc. (Damiani et ah, 2011). AR is generally implemented through various 
different approaches and platforms such as location based AR systems, indoor applications and edutainment 
(Krevelen&Poelman, 2010). 

One of the current challenges for AR technology is to implement effective AR on mobile platforms. 
Mobile AR has become a most recent trend in location based services and interactive graphic applications 
that allow users to experience visualization and interaction with 3D models or contents on mobile devices. 
Currently, mobile AR has also been implemented efficiently in various innovative applications such as 
gaming, shopping guides, advertising, edutainment, travel guides, museum guides and medical visualization 
(Papagiannakis et ah, 2008). These applications can be enhanced by adapting the visualization, tracking, 
recognition, interaction, displays and user interface techniques with real world scenes and virtual 
environments (Zhou et al., 2008)(Pery et al., 2011). 

Nowadays, AR technology has become a powerful tool to enhance education and learning experiences by 
visualizing 3D virtual contents on mobile technology (Kesim&Ozarslan, 2012)(Wu et ah, 2013). Students are 
able to view digital contents augmented in real scenes using tracking techniques or even current location on 
mobile devices such as a smartphone or tablet. Interaction between students and digital contents can also be 
done on AR environments for indoor or outdoor uses. To do so, educational content providers or teachers can 
design what they want their students to learn the physical objects and augmented digital contents. On the 
other hand, it would be a useful and innovative way to extensively enhance students’ learning experiences by 
providing a tool (mobile AR client) to create their AR learning environments. Students could personalize 
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their AR learning environment by selecting preferred contents (e.g. by age, gender, etc.) associated with the 
reference objects tracked in the real world. Content service providers provide these digital contents, which 
are then displayed in AR environment — the resulting AR environment can be saved and reused for 
presentation again in other different learning experiences. 

There are some tools that enable mobile users to create their own AR channels for indoor and outdoor 
environments such as Junaio (www.junaio.com), Layar (www.layar.com) and Aurasma (www.aurasma.com). 
These applications enable mobile users to create AR environments and save them into their channels on the 
cloud server. This technique is useful because it allows general mobile users who don’t want to or who can’t 
develop mobile AR applications to have their own AR environments. However, these AR applications still 
have some limitations because they are implemented on closed platforms such that a user’s AR environment 
can only be retrieved via the commercial application, i.e. you cannot reuse a Junaio environment in an 
Aurasma environment. Moreover, most commercial mobile indoor AR applications provide users with 
limited amounts of data or contents for augmenting real world scenes. There is no communication channel 
for current AR applications (e.g. Junaio, Layar, Aurasma or specific research application like an AR game, 
etc.) that allows them to download or obtain dynamic contents from other third party data sources in real-time 
— our solution proposes an open model for accessing contents via web services. Hence, there is no content 
sharing between mobile users and service providers limiting digital content acquisition to the mobile AR 
application builder’s pre-design contents built into their applications. 

This paper presents the design of a multiple object tracking AR architecture on a mobile platform that 
exploits service-orientation to access digital media contents that could be beneficially applied in learning and 
education scenarios. Our architecture provides a mobile AR learning environment with access to other digital 
content generation services such as photogrammetry, or third party content requests, in order to obtain 
associated 3D or digital media contents. Such contents, selected dynamically and based on multiple object 
tracking, provide for novel learning scenarios, as students will be able to create personal AR environments on 
real scenes for each different learning situations by utilizing camera-embedded mobile devices. 


2. SERVICE ORIENTED MOBILE AUGMENTED REALITY SYSTEM 

Our mobile AR system exploits the concept of service orientation to implement its software architecture as a 
designed pattern based on distributed system components that provide application functionality via a set of 
web services (http://msdn.microsoft.com/en-us/library/bb972929.aspx). The service-oriented architecture is 
largely composed of 3 components: the mobile multiple object tracking AR client (web service calls), the 
web service framework itself, and participating dynamic digital content service providers (web service calls). 
The web service framework is designed to link and manage connections between the mobile AR client and 
web services content providers over wireless or mobile network. Mobile users will utilize the mobile AR 
client in order to perform general AR tasks such as multiple object tracking, interaction and content 
visualization. Moreover, users can also request for additional contents and services from open service 
providers such as a photogrammetry service so that they can decide what to place and manipulate on the AR 
learning environments. Figure 1 shows the service-oriented architecture for a mobile multiple object tracking 
AR system and a test screenshot illustrating the AR application on an iPhone 5, which utilizing the Metaio 
AR SDK (www.metaio.com/sdk/) to track the 3D biscuit tube container. Once the biscuit tube is tracked, 3 
pieces of associated digital media data (a 3D model, a price tag and an image) augment the real world scene. 
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Figure 1. Service-oriented architecture with multiple objects tracking on a mobile AR system 

Other example learning scenarios could involve utilizing the video frames taken of the physical 3D object 
(e.g. the biscuit tube) to request a 3D model build via a photogrammetry service for later use by the learner. 


3. WEB SERVICE FRAMEWORK 

Integrating a service oriented architecture and web service framework into any application generally provides 
more value and usability for the platform of applications (devices, architectures, software), and this includes 
mobile AR platforms (Belimpasakis et al, 2012)(Wang&Wang, 2008). Interoperable AR applications are 
projected to present virtual media or digital contents on see-through AR browsers, which can be used instead 
of web browsers in the outdoor AR environment (Lee, 2009)(Selonen, 2012). Open and interoperable 
networks are based on service-oriented architectures, and consequently can be accessed over the Internet 
through mobile network and client-server architectures. Examples of web services, which developers can 
easily access to generate platform independent digital contents, include Web Map Services, mash-up 
services, geospatial and social network data, 3D models, etc. 

In this research, the web service framework is combined into a mobile AR application and set of example 
web service providers to provide service interfaces and connections to appropriate digital contents for the 
mobile AR client. In a mobile AR learning environment, the mobile client is able to request services and 
receive related dynamic media contents for viewing and manipulating in the AR environment, see screen shot 
in Figure 1. 


4. EDUCATION AND LEARNING SCENARIOS 

We designed the multiple objects tracking AR client as an application on mobile devices, where the web 
service framework will work as a back-end system. Our goal is to create a mobile AR client with a service- 
oriented architecture that is adaptable and will be able to support various mobile AR leaning scenarios. There 
are interesting issues that arise when considering mobile AR system and service-orientation: 

• Sharing learning materials such as animation, 2D/3D models, images or videos over the Internet. 
Students will be able to download and place their prospective contents on AR environments by 
identifying marker or markerless (e.g. 3D) objects to each real world scene. Additionally, newly 
acquired contents integrated in existing AR environments could then be visualized in other 
situations. 

• General mobile AR applications mainly require tracking, rendering and visualization tasks, for 
which the final outcome of a reference or tracked object will be revealed on the screen. Existing 
tracking techniques that would be efficiently developed on mobile AR platform are marker, 
markerless (2D and 3D) and location based tracking. Markerless tracking augments digital contents 
on top of real scenes where mobile users will see a combination between computer-generated media 
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contents and the real environment. In addition, 3D tracking, see Figure 1, is outstanding in its ability 
to visualizes virtual contents augmented with the 3D object tracked — this greatly increases the 
application range for AR scenarios, one can imagine for example a museum’s physical artifact or 
object being augmented with associated dynamic media objects. 

• A good illustration of a learning scenario for our mobile service oriented AR platform would be 
where a museum visitor wishes to access further information about a physical artifact on display. In 
such a scenario, typically the museum’s artifact will only have a card presenting the curators 
interpretation of the artifact. The visitor is able to use either the card itself as a trigger image to 
reveal more augmented contents, but more interestingly they will also be able to take several images 
of the object and use a photogrammetry service to reconstruct the object in 3D and then personalize 
that object with the existing augmented contents for visualizing and learning outside the museum, 
thus generating their own interactive learning preferences by selecting, manipulating and placing 
preferred contents on AR environments. 


5. AUGMENTED REALITY IN LEARNING ENVIRONMENTS 

AR techniques can enhance students’ learning experiences by presenting media contents overlaid on 
reference markers or markerless tracking objects. Students will be able to learn from augmented media 
contents relative to the real environments objects, which is more interesting and adaptable than reading books 
or viewing images. This paper proposes the application of AR technology on a mobile platform and web 
service architecture, which efficiently enhances education and learning scenarios by offering some features 
for students to acquire new and various contents from appropriate services and also create their own AR 
learning environments. The following issues improvements to current AR systems through a service- 
orientation integrated with the learning environments. 

• General AR applications or even AR in education are often offered in closed platforms that enable 
users to only view AR contents overlaid on a marker. The contents could be 3D models, images, 
videos, etc. that have relevance to lessons or learning scenarios that the teachers or content 
providers want to offer. In contrast, in our system, we design an open AR web service architecture 
on a mobile platform that allows users to acquire and view more associated contents from open 
service providers. 

• Multiple object tracking is a feature in our system that enable users to track 3D objects rather than 
tracking markers or markerless such as 2D images or photographs, which is typically found in 
general AR applications. This will improve learning techniques by augmenting real objects, offer 
new views, and allow the users to acquire associated contents in order to learn through AR 
environments. 

• Photogrammetry services and AR environment personalization are also features that allow users to 
acquire 3D models from real objects by requesting image-based reconstruction services, which can 
then also be used to create personal AR environments for taking away and viewing in different 
situations. These features cannot be found in typical AR applications for education or learning up to 
now — students currently are not be able to obtain new associated media contents or save and 
manipulate offered AR environments. Therefore, the learning processes will be enhanced from 
small scale to large-scale AR environments by utilizing our mobile web service AR architecture. 


6. CONCLUSION 

The service-oriented mobile multiple object tracking AR system is designed to efficiently support a web 
service connection and AR environment manipulation feature for mobile AR applications, which could 
benefit learning and education scenarios. The proposed mobile AR application could provide students with 
an AR web service framework allowing them to request dynamic digital media contents from open source 
digital content providers. Currently, the architecture can detect multiple 3D objects and assign multiple 
digital contents to the detected object in the AR environment. It is also able to connect to a web service 
provider via wireless/mobile network in order to request associated media contents and overlay those on the 
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real scene. The next steps are to detect multiple objects in parallel, add meaning to multiple objects thus 
detected and so modify the augmented digital contents based on the meaning of, say two objects tracked, and 
so on. Utilizing third party contents through this architecture’s web service has many obvious benefits over 
current closed commercial AR systems. 
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